Implement Config Inversion with Default Strictness of `Warning` #9539

mhlidd · 2025-09-15T21:29:37Z

What Does This Do

This PR implements basic Config Inversion, aiming to document all DD/OTEL environment variables used in the repo in a supported-configurations.json file.

Components:

ConfigInversionStrictStyle.java
- Enum for how strict to run Config Inversion with.
- Options are Strict which does not allow any usage of an unknown DD/OTEL environment variable, Warning which allows the usage but sends telemetry about unknown environment variables, and Test which allows the usage of unknown environment variables without sending telemetry.
- Default value for Strictness is Warning
ConfigHelper.java
- Layer of abstraction between Environment Variable Source and Environment Component that directly compares data loaded from supported-configurations.json to ensure the environment variable queried for is "known"
- Determines what behavior to take given the ConfigInversionStrictStyle that is set.

Motivation

This PR supports the general Config Inversion theme that has already been implemented in dd-trace-js and currently being implemented in dd-trace-go and dd-trace-rb. Here is the RFC that documents what this project entails.

Additional Notes

This is a follow-up to

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-09-15T22:10:22Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mhlidd/config_inversion_base
git_commit_date	1758822615	1758829485
git_commit_sha	`61c2d97`	`8ba684e`
release_version	1.54.0-SNAPSHOT~61c2d97420	1.54.0-SNAPSHOT~8ba684e640

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1758831335	1758831335
ci_job_id	1148626949	1148626949
ci_pipeline_id	77581774	77581774
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-8ql0chrp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-8ql0chrp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 4 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.54.0-SNAPSHOT~8ba684e640, baseline=1.54.0-SNAPSHOT~61c2d97420

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.014 s) : 0, 1014475
Total [baseline] (8.674 s) : 0, 8673701
Agent [candidate] (1.008 s) : 0, 1007856
Total [candidate] (8.706 s) : 0, 8706350
section iast
Agent [baseline] (1.145 s) : 0, 1144858
Total [baseline] (9.277 s) : 0, 9276952
Agent [candidate] (1.152 s) : 0, 1151752
Total [candidate] (9.285 s) : 0, 9284912

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.014 s	-
Agent	iast	1.145 s	130.383 ms (12.9%)
Total	tracing	8.674 s	-
Total	iast	9.277 s	603.251 ms (7.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.008 s	-
Agent	iast	1.152 s	143.896 ms (14.3%)
Total	tracing	8.706 s	-
Total	iast	9.285 s	578.562 ms (6.6%)

gantt
    title insecure-bank - break down per module: candidate=1.54.0-SNAPSHOT~8ba684e640, baseline=1.54.0-SNAPSHOT~61c2d97420

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.475 ms) : 0, 1475
crashtracking [candidate] (1.455 ms) : 0, 1455
BytebuddyAgent [baseline] (694.782 ms) : 0, 694782
BytebuddyAgent [candidate] (689.628 ms) : 0, 689628
GlobalTracer [baseline] (249.6 ms) : 0, 249600
GlobalTracer [candidate] (248.762 ms) : 0, 248762
AppSec [baseline] (31.145 ms) : 0, 31145
AppSec [candidate] (30.983 ms) : 0, 30983
Debugger [baseline] (6.407 ms) : 0, 6407
Debugger [candidate] (6.369 ms) : 0, 6369
Remote Config [baseline] (691.389 µs) : 0, 691
Remote Config [candidate] (684.026 µs) : 0, 684
Telemetry [baseline] (9.056 ms) : 0, 9056
Telemetry [candidate] (8.893 ms) : 0, 8893
section iast
crashtracking [baseline] (1.463 ms) : 0, 1463
crashtracking [candidate] (1.453 ms) : 0, 1453
BytebuddyAgent [baseline] (809.932 ms) : 0, 809932
BytebuddyAgent [candidate] (815.344 ms) : 0, 815344
GlobalTracer [baseline] (237.172 ms) : 0, 237172
GlobalTracer [candidate] (238.555 ms) : 0, 238555
IAST [baseline] (26.033 ms) : 0, 26033
IAST [candidate] (26.122 ms) : 0, 26122
AppSec [baseline] (33.932 ms) : 0, 33932
AppSec [candidate] (34.046 ms) : 0, 34046
Debugger [baseline] (6.047 ms) : 0, 6047
Debugger [candidate] (6.089 ms) : 0, 6089
Remote Config [baseline] (593.77 µs) : 0, 594
Remote Config [candidate] (607.421 µs) : 0, 607
Telemetry [baseline] (8.419 ms) : 0, 8419
Telemetry [candidate] (8.426 ms) : 0, 8426

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.54.0-SNAPSHOT~8ba684e640, baseline=1.54.0-SNAPSHOT~61c2d97420

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.006 s) : 0, 1005769
Total [baseline] (10.672 s) : 0, 10672246
Agent [candidate] (1.005 s) : 0, 1004798
Total [candidate] (10.7 s) : 0, 10699587
section appsec
Agent [baseline] (1.185 s) : 0, 1185118
Total [baseline] (10.975 s) : 0, 10975060
Agent [candidate] (1.199 s) : 0, 1198632
Total [candidate] (10.963 s) : 0, 10963057
section iast
Agent [baseline] (1.146 s) : 0, 1145919
Total [baseline] (10.953 s) : 0, 10953235
Agent [candidate] (1.145 s) : 0, 1144850
Total [candidate] (10.974 s) : 0, 10973522
section profiling
Agent [baseline] (1.154 s) : 0, 1154240
Total [baseline] (11.032 s) : 0, 11032086
Agent [candidate] (1.149 s) : 0, 1149474
Total [candidate] (11.047 s) : 0, 11046664

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.006 s	-
Agent	appsec	1.185 s	179.349 ms (17.8%)
Agent	iast	1.146 s	140.15 ms (13.9%)
Agent	profiling	1.154 s	148.471 ms (14.8%)
Total	tracing	10.672 s	-
Total	appsec	10.975 s	302.814 ms (2.8%)
Total	iast	10.953 s	280.989 ms (2.6%)
Total	profiling	11.032 s	359.839 ms (3.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.005 s	-
Agent	appsec	1.199 s	193.834 ms (19.3%)
Agent	iast	1.145 s	140.052 ms (13.9%)
Agent	profiling	1.149 s	144.676 ms (14.4%)
Total	tracing	10.7 s	-
Total	appsec	10.963 s	263.47 ms (2.5%)
Total	iast	10.974 s	273.935 ms (2.6%)
Total	profiling	11.047 s	347.077 ms (3.2%)

gantt
    title petclinic - break down per module: candidate=1.54.0-SNAPSHOT~8ba684e640, baseline=1.54.0-SNAPSHOT~61c2d97420

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.456 ms) : 0, 1456
crashtracking [candidate] (1.454 ms) : 0, 1454
BytebuddyAgent [baseline] (688.47 ms) : 0, 688470
BytebuddyAgent [candidate] (687.117 ms) : 0, 687117
GlobalTracer [baseline] (247.925 ms) : 0, 247925
GlobalTracer [candidate] (248.099 ms) : 0, 248099
AppSec [baseline] (30.747 ms) : 0, 30747
AppSec [candidate] (31.009 ms) : 0, 31009
Debugger [baseline] (6.354 ms) : 0, 6354
Debugger [candidate] (6.382 ms) : 0, 6382
Remote Config [baseline] (671.284 µs) : 0, 671
Remote Config [candidate] (686.526 µs) : 0, 687
Telemetry [baseline] (8.963 ms) : 0, 8963
Telemetry [candidate] (8.949 ms) : 0, 8949
section appsec
crashtracking [baseline] (1.451 ms) : 0, 1451
crashtracking [candidate] (1.489 ms) : 0, 1489
BytebuddyAgent [baseline] (710.308 ms) : 0, 710308
BytebuddyAgent [candidate] (719.216 ms) : 0, 719216
GlobalTracer [baseline] (239.69 ms) : 0, 239690
GlobalTracer [candidate] (242.484 ms) : 0, 242484
AppSec [baseline] (173.036 ms) : 0, 173036
AppSec [candidate] (172.636 ms) : 0, 172636
Debugger [baseline] (5.925 ms) : 0, 5925
Debugger [candidate] (6.844 ms) : 0, 6844
Remote Config [baseline] (636.367 µs) : 0, 636
Remote Config [candidate] (658.216 µs) : 0, 658
Telemetry [baseline] (8.409 ms) : 0, 8409
Telemetry [candidate] (9.312 ms) : 0, 9312
IAST [baseline] (24.574 ms) : 0, 24574
IAST [candidate] (24.76 ms) : 0, 24760
section iast
crashtracking [baseline] (1.465 ms) : 0, 1465
crashtracking [candidate] (1.462 ms) : 0, 1462
BytebuddyAgent [baseline] (810.194 ms) : 0, 810194
BytebuddyAgent [candidate] (809.905 ms) : 0, 809905
GlobalTracer [baseline] (237.767 ms) : 0, 237767
GlobalTracer [candidate] (237.266 ms) : 0, 237266
AppSec [baseline] (33.7 ms) : 0, 33700
AppSec [candidate] (33.941 ms) : 0, 33941
Debugger [baseline] (6.144 ms) : 0, 6144
Debugger [candidate] (6.074 ms) : 0, 6074
Remote Config [baseline] (596.414 µs) : 0, 596
Remote Config [candidate] (602.899 µs) : 0, 603
Telemetry [baseline] (8.434 ms) : 0, 8434
Telemetry [candidate] (8.445 ms) : 0, 8445
IAST [baseline] (26.226 ms) : 0, 26226
IAST [candidate] (25.979 ms) : 0, 25979
section profiling
ProfilingAgent [baseline] (101.21 ms) : 0, 101210
ProfilingAgent [candidate] (101.126 ms) : 0, 101126
crashtracking [baseline] (1.453 ms) : 0, 1453
crashtracking [candidate] (1.422 ms) : 0, 1422
BytebuddyAgent [baseline] (719.677 ms) : 0, 719677
BytebuddyAgent [candidate] (717.55 ms) : 0, 717550
GlobalTracer [baseline] (224.317 ms) : 0, 224317
GlobalTracer [candidate] (222.46 ms) : 0, 222460
AppSec [baseline] (31.251 ms) : 0, 31251
AppSec [candidate] (31.221 ms) : 0, 31221
Debugger [baseline] (6.485 ms) : 0, 6485
Debugger [candidate] (7.242 ms) : 0, 7242
Remote Config [baseline] (710.786 µs) : 0, 711
Remote Config [candidate] (713.472 µs) : 0, 713
Telemetry [baseline] (16.138 ms) : 0, 16138
Telemetry [candidate] (15.523 ms) : 0, 15523
Profiling [baseline] (102.447 ms) : 0, 102447
Profiling [candidate] (101.726 ms) : 0, 101726

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mhlidd/config_inversion_base
git_commit_date	1758822615	1758829485
git_commit_sha	`61c2d97`	`8ba684e`
release_version	1.54.0-SNAPSHOT~61c2d97420	1.54.0-SNAPSHOT~8ba684e640

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1758831092	1758831092
ci_job_id	1148626950	1148626950
ci_pipeline_id	77581774	77581774
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-nku9itpt 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-nku9itpt 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 5 performance improvements and 1 performance regressions! Performance is the same for 6 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:iast:high_load	worse [+571.688µs; +942.240µs] or [+5.878%; +9.688%]	unstable [-91.447op/s; +23.009op/s] or [-19.156%; +4.820%]	10.483ms	443.156op/s	9.726ms	477.375op/s
scenario:load:insecure-bank:no_agent:high_load	better [-222.129µs; -113.312µs] or [-5.098%; -2.600%]	unstable [-86.656op/s; +168.969op/s] or [-8.229%; +16.045%]	4.190ms	1094.250op/s	4.357ms	1053.094op/s
scenario:load:insecure-bank:iast_GLOBAL:high_load	better [-991.085µs; -587.991µs] or [-9.032%; -5.358%]	unstable [-22.494op/s; +86.994op/s] or [-5.307%; +20.525%]	10.184ms	456.094op/s	10.974ms	423.844op/s
scenario:load:insecure-bank:iast_FULL:high_load	better [-1.342ms; -0.721ms] or [-9.078%; -4.875%]	unstable [-14.597op/s; +61.659op/s] or [-4.631%; +19.563%]	13.756ms	338.719op/s	14.788ms	315.188op/s
scenario:load:petclinic:no_agent:high_load	better [-2.011ms; -1.395ms] or [-5.489%; -3.809%]	unstable [-3.778op/s; +12.973op/s] or [-2.924%; +10.041%]	34.929ms	133.800op/s	36.632ms	129.203op/s
scenario:load:petclinic:profiling:high_load	better [-2.730ms; -1.778ms] or [-5.579%; -3.634%]	unstable [-2.651op/s; +11.826op/s] or [-2.771%; +12.363%]	46.674ms	100.237op/s	48.928ms	95.650op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.54.0-SNAPSHOT~8ba684e640, baseline=1.54.0-SNAPSHOT~61c2d97420
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.632 ms) : 36336, 36927
.   : milestone, 36632,
appsec (46.97 ms) : 46555, 47385
.   : milestone, 46970,
code_origins (44.181 ms) : 43794, 44568
.   : milestone, 44181,
iast (44.987 ms) : 44614, 45360
.   : milestone, 44987,
profiling (48.928 ms) : 48464, 49392
.   : milestone, 48928,
tracing (45.242 ms) : 44849, 45636
.   : milestone, 45242,
section candidate
no_agent (34.929 ms) : 34652, 35205
.   : milestone, 34929,
appsec (47.357 ms) : 46949, 47765
.   : milestone, 47357,
code_origins (43.0 ms) : 42636, 43363
.   : milestone, 43000,
iast (44.178 ms) : 43808, 44548
.   : milestone, 44178,
profiling (46.674 ms) : 46255, 47093
.   : milestone, 46674,
tracing (44.63 ms) : 44260, 45000
.   : milestone, 44630,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.632 ms [36.336 ms, 36.927 ms]	-
appsec	46.97 ms [46.555 ms, 47.385 ms]	10.338 ms (28.2%)
code_origins	44.181 ms [43.794 ms, 44.568 ms]	7.55 ms (20.6%)
iast	44.987 ms [44.614 ms, 45.36 ms]	8.355 ms (22.8%)
profiling	48.928 ms [48.464 ms, 49.392 ms]	12.296 ms (33.6%)
tracing	45.242 ms [44.849 ms, 45.636 ms]	8.611 ms (23.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	34.929 ms [34.652 ms, 35.205 ms]	-
appsec	47.357 ms [46.949 ms, 47.765 ms]	12.429 ms (35.6%)
code_origins	43.0 ms [42.636 ms, 43.363 ms]	8.071 ms (23.1%)
iast	44.178 ms [43.808 ms, 44.548 ms]	9.249 ms (26.5%)
profiling	46.674 ms [46.255 ms, 47.093 ms]	11.745 ms (33.6%)
tracing	44.63 ms [44.26 ms, 45.0 ms]	9.702 ms (27.8%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.54.0-SNAPSHOT~8ba684e640, baseline=1.54.0-SNAPSHOT~61c2d97420
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.357 ms) : 4303, 4412
.   : milestone, 4357,
iast (9.726 ms) : 9558, 9893
.   : milestone, 9726,
iast_FULL (14.788 ms) : 14491, 15085
.   : milestone, 14788,
iast_GLOBAL (10.974 ms) : 10776, 11171
.   : milestone, 10974,
profiling (9.277 ms) : 9113, 9440
.   : milestone, 9277,
tracing (7.916 ms) : 7787, 8045
.   : milestone, 7916,
section candidate
no_agent (4.19 ms) : 4143, 4236
.   : milestone, 4190,
iast (10.483 ms) : 10306, 10660
.   : milestone, 10483,
iast_FULL (13.756 ms) : 13476, 14036
.   : milestone, 13756,
iast_GLOBAL (10.184 ms) : 10008, 10361
.   : milestone, 10184,
profiling (9.272 ms) : 9125, 9418
.   : milestone, 9272,
tracing (7.766 ms) : 7657, 7875
.   : milestone, 7766,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.357 ms [4.303 ms, 4.412 ms]	-
iast	9.726 ms [9.558 ms, 9.893 ms]	5.368 ms (123.2%)
iast_FULL	14.788 ms [14.491 ms, 15.085 ms]	10.43 ms (239.4%)
iast_GLOBAL	10.974 ms [10.776 ms, 11.171 ms]	6.616 ms (151.8%)
profiling	9.277 ms [9.113 ms, 9.44 ms]	4.919 ms (112.9%)
tracing	7.916 ms [7.787 ms, 8.045 ms]	3.558 ms (81.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.19 ms [4.143 ms, 4.236 ms]	-
iast	10.483 ms [10.306 ms, 10.66 ms]	6.293 ms (150.2%)
iast_FULL	13.756 ms [13.476 ms, 14.036 ms]	9.566 ms (228.3%)
iast_GLOBAL	10.184 ms [10.008 ms, 10.361 ms]	5.994 ms (143.1%)
profiling	9.272 ms [9.125 ms, 9.418 ms]	5.082 ms (121.3%)
tracing	7.766 ms [7.657 ms, 7.875 ms]	3.576 ms (85.4%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mhlidd/config_inversion_base
git_commit_date	1758822615	1758829485
git_commit_sha	`61c2d97`	`8ba684e`
release_version	1.54.0-SNAPSHOT~61c2d97420	1.54.0-SNAPSHOT~8ba684e640

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1758831563	1758831563
ci_job_id	1148626951	1148626951
ci_pipeline_id	77581774	77581774
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-2-kayq9c2m 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-2-kayq9c2m 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.54.0-SNAPSHOT~8ba684e640, baseline=1.54.0-SNAPSHOT~61c2d97420
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.478 ms) : 1467, 1490
.   : milestone, 1478,
appsec (3.652 ms) : 3440, 3864
.   : milestone, 3652,
iast (2.21 ms) : 2147, 2273
.   : milestone, 2210,
iast_GLOBAL (2.251 ms) : 2188, 2315
.   : milestone, 2251,
profiling (2.054 ms) : 2003, 2105
.   : milestone, 2054,
tracing (2.024 ms) : 1975, 2073
.   : milestone, 2024,
section candidate
no_agent (1.481 ms) : 1469, 1492
.   : milestone, 1481,
appsec (3.748 ms) : 3530, 3966
.   : milestone, 3748,
iast (2.201 ms) : 2138, 2264
.   : milestone, 2201,
iast_GLOBAL (2.256 ms) : 2193, 2320
.   : milestone, 2256,
profiling (2.056 ms) : 2006, 2107
.   : milestone, 2056,
tracing (2.023 ms) : 1974, 2072
.   : milestone, 2023,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.478 ms [1.467 ms, 1.49 ms]	-
appsec	3.652 ms [3.44 ms, 3.864 ms]	2.174 ms (147.0%)
iast	2.21 ms [2.147 ms, 2.273 ms]	731.404 µs (49.5%)
iast_GLOBAL	2.251 ms [2.188 ms, 2.315 ms]	772.967 µs (52.3%)
profiling	2.054 ms [2.003 ms, 2.105 ms]	575.266 µs (38.9%)
tracing	2.024 ms [1.975 ms, 2.073 ms]	545.996 µs (36.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.481 ms [1.469 ms, 1.492 ms]	-
appsec	3.748 ms [3.53 ms, 3.966 ms]	2.268 ms (153.2%)
iast	2.201 ms [2.138 ms, 2.264 ms]	720.4 µs (48.7%)
iast_GLOBAL	2.256 ms [2.193 ms, 2.32 ms]	775.647 µs (52.4%)
profiling	2.056 ms [2.006 ms, 2.107 ms]	575.835 µs (38.9%)
tracing	2.023 ms [1.974 ms, 2.072 ms]	542.557 µs (36.6%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.54.0-SNAPSHOT~8ba684e640, baseline=1.54.0-SNAPSHOT~61c2d97420
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.735 s) : 14735000, 14735000
.   : milestone, 14735000,
appsec (15.174 s) : 15174000, 15174000
.   : milestone, 15174000,
iast (18.458 s) : 18458000, 18458000
.   : milestone, 18458000,
iast_GLOBAL (18.008 s) : 18008000, 18008000
.   : milestone, 18008000,
profiling (15.317 s) : 15317000, 15317000
.   : milestone, 15317000,
tracing (15.034 s) : 15034000, 15034000
.   : milestone, 15034000,
section candidate
no_agent (15.47 s) : 15470000, 15470000
.   : milestone, 15470000,
appsec (14.935 s) : 14935000, 14935000
.   : milestone, 14935000,
iast (18.359 s) : 18359000, 18359000
.   : milestone, 18359000,
iast_GLOBAL (18.266 s) : 18266000, 18266000
.   : milestone, 18266000,
profiling (14.945 s) : 14945000, 14945000
.   : milestone, 14945000,
tracing (15.095 s) : 15095000, 15095000
.   : milestone, 15095000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.735 s [14.735 s, 14.735 s]	-
appsec	15.174 s [15.174 s, 15.174 s]	439.0 ms (3.0%)
iast	18.458 s [18.458 s, 18.458 s]	3.723 s (25.3%)
iast_GLOBAL	18.008 s [18.008 s, 18.008 s]	3.273 s (22.2%)
profiling	15.317 s [15.317 s, 15.317 s]	582.0 ms (3.9%)
tracing	15.034 s [15.034 s, 15.034 s]	299.0 ms (2.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.47 s [15.47 s, 15.47 s]	-
appsec	14.935 s [14.935 s, 14.935 s]	-535.0 ms (-3.5%)
iast	18.359 s [18.359 s, 18.359 s]	2.889 s (18.7%)
iast_GLOBAL	18.266 s [18.266 s, 18.266 s]	2.796 s (18.1%)
profiling	14.945 s [14.945 s, 14.945 s]	-525.0 ms (-3.4%)
tracing	15.095 s [15.095 s, 15.095 s]	-375.0 ms (-2.4%)

datadog-official · 2025-09-16T18:23:19Z

🎯 Code Coverage
• Patch Coverage: 79.45%
• Total Coverage: 57.84% (+0.45%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 8ba684e | Docs | Was this helpful? Give us feedback!}

github-actions · 2025-09-18T15:13:54Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

PerfectSlayer

👏 praise: ‏Thanks for splitting the PR related to your config changes, that really helps to review 👍

🎯 suggestion: ‏I would recommend trying to decrease the overall (cognitive) complexity by refactoring using new few methods to de-duplicate code and give more meaning.

For example this should have its own function with a meaningful name:

if (key.startsWith("DD_")
          || key.startsWith("OTEL_")
          || configSource.getAliasMapping().containsKey(key))

PerfectSlayer · 2025-09-19T12:15:16Z

utils/config-utils/src/main/java/datadog/trace/config/inversion/ConfigHelper.java

+import java.util.List;
+import java.util.Map;
+
+public class ConfigHelper {


🎯 suggestion: ‏Helper class usually are final and have a private constructor.

I was also wondering if this could be made into a singleton as well... and if so there would be no reason to have static methods under it. Not sure if that is the right or wrong direction to go in

Usually, an helper class does not have a state. So if you have a state but only one instance, that would be a singleton.
Here might be a big ambiguous because most of the state is immutable, right? It's more like a big static data table and calling the helper methods won't mutate it. But as you have the StrictStyle thing that can change the behavior, I would with a singleton.

Keep in mind helper class / methods should be easier to test than singleton that usually are painful to deal with on test scenario.

utils/config-utils/src/main/java/datadog/trace/config/inversion/ConfigInversionStrictStyle.java

utils/config-utils/src/main/java/datadog/trace/config/inversion/ConfigHelper.java

PerfectSlayer · 2025-09-19T12:27:01Z

utils/config-utils/src/main/java/datadog/trace/config/inversion/ConfigHelper.java

+                  + (configSource.getAliasMapping().containsKey(key)
+                      ? "Please use " + configSource.getAliasMapping().get(key) + " instead."
+                      : configSource.getDeprecatedConfigurations().get(key));
+          System.err.println(warning);


/

❔ question: ‏Is it expected to log on System.err?

I changed it to a log just now. (I remember there was something weird with logging when I first implemented but that might have been due to something separate.)

Yes, there is issue with the initializing the logging framework too early. This will depend on the usage that's done of the config helper. It might break in bootstrap for example, while reporting to system.err won't be great once we bootstrap.
There might be some new design constraints here... 🤔

This would very possibly be an issue, given that this class will likely be invoked at bootstrap, and initialize the logger.

I agree that keeping system.err is suboptimal. One option is to not use the ConfigHelper at all in bootstrap, which we could do unless there is a better solution...

@PerfectSlayer Stuart seemed to be in agreement that avoiding ConfigHelper in bootstrap during the migration to ConfigHelper is viable since few configs are used there and are rarely changed. Using logging here should be fine for this PR, and I'll take note to not migrate bootstrap for the followup PR

utils/config-utils/src/main/java/datadog/trace/config/inversion/ConfigHelper.java

utils/config-utils/src/main/java/datadog/trace/config/inversion/ConfigInversionStrictStyle.java

utils/config-utils/src/main/java/datadog/trace/config/inversion/ConfigHelper.java

…adability and hide supported-configuration details

… ControllableEnvironmentVariables

pr-commenter · 2025-09-25T01:00:06Z

Debugger benchmarks

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
ci_job_date	1758830170	1758830516
end_time	2025-09-25T19:57:31	2025-09-25T20:03:16
git_branch	master	mhlidd/config_inversion_base
git_commit_sha	`61c2d97`	`8ba684e`
start_time	2025-09-25T19:56:11	2025-09-25T20:01:57

See matching parameters

	Baseline	Candidate
ci_job_id	1148626956	1148626956
ci_pipeline_id	77581774	77581774
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date	1758829485	1758829485

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 5 unstable metrics.

See unchanged results

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:noprobe	unstable [-31.560µs; +14.449µs] or [-11.225%; +5.139%]	unstable [-43.694µs; +22.045µs] or [-13.504%; +6.813%]	unstable [-59.563µs; +29.459µs] or [-17.503%; +8.657%]	unstable [-116.140µs; +100.503µs] or [-12.026%; +10.407%]	same
scenario:basic	same	same	same	unstable [-18.318µs; +183.626µs] or [-2.385%; +23.913%]	same
scenario:loop	unsure [-9.394µs; -2.987µs] or [-0.106%; -0.034%]	unsure [-15.221µs; -4.500µs] or [-0.170%; -0.050%]	same	same	same

Request duration reports for reports

gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (323.559 µs) : 284, 363
.   : milestone, 324,
basic (281.378 µs) : 275, 288
.   : milestone, 281,
loop (8.969 ms) : 8964, 8974
.   : milestone, 8969,
section candidate
noprobe (312.735 µs) : 292, 333
.   : milestone, 313,
basic (280.181 µs) : 274, 287
.   : milestone, 280,
loop (8.959 ms) : 8955, 8964
.   : milestone, 8959,

baseline results

Scenario	Request median duration [CI 0.99]
noprobe	323.559 µs [284.341 µs, 362.777 µs]
basic	281.378 µs [274.852 µs, 287.904 µs]
loop	8.969 ms [8.964 ms, 8.974 ms]

candidate results

Scenario	Request median duration [CI 0.99]
noprobe	312.735 µs [292.23 µs, 333.24 µs]
basic	280.181 µs [273.754 µs, 286.608 µs]
loop	8.959 ms [8.955 ms, 8.964 ms]

PerfectSlayer

Looking good.

Left a comment about unnecessary allocation.

...fig-utils/src/test/java/datadog/trace/config/inversion/TestSupportedConfigurationSource.java

mhlidd force-pushed the mhlidd/config_inversion_base branch 2 times, most recently from 1a576ea to 512b2c6 Compare September 15, 2025 21:34

mhlidd changed the title ~~Mhlidd/config inversion base~~ Implement Config Inversion with Default Strictness of Warning Sep 17, 2025

mhlidd marked this pull request as ready for review September 18, 2025 15:13

mhlidd requested review from a team as code owners September 18, 2025 15:13

mhlidd requested review from AlexeyKuznetsov-DD and mcculls and removed request for a team September 18, 2025 15:13

mhlidd added type: enhancement Enhancements and improvements type: documentation comp: api Tracer public API labels Sep 18, 2025

mhlidd requested a review from PerfectSlayer September 18, 2025 15:15

mhlidd mentioned this pull request Sep 18, 2025

Adding supported-configurations.json file #9563

Merged

mhlidd changed the base branch from master to mhlidd/config_inversion_plugins September 18, 2025 21:25

mhlidd force-pushed the mhlidd/config_inversion_base branch from aa18112 to 606dfd0 Compare September 18, 2025 21:32

mhlidd changed the base branch from mhlidd/config_inversion_plugins to mhlidd/migrate_config-utils_tests September 18, 2025 22:06

mhlidd force-pushed the mhlidd/config_inversion_base branch from 606dfd0 to 7cea99b Compare September 18, 2025 22:09

PerfectSlayer reviewed Sep 19, 2025

View reviewed changes

utils/config-utils/src/main/java/datadog/trace/config/inversion/ConfigInversionStrictStyle.java Outdated Show resolved Hide resolved

dougqh reviewed Sep 19, 2025

View reviewed changes

utils/config-utils/src/main/java/datadog/trace/config/inversion/ConfigHelper.java Outdated Show resolved Hide resolved

bric3 mentioned this pull request Sep 23, 2025

Adding Gradle Plugins for Config Inversion #9565

Merged

mhlidd force-pushed the mhlidd/migrate_config-utils_tests branch from 57483a2 to 1574d9b Compare September 23, 2025 21:02

mhlidd force-pushed the mhlidd/config_inversion_base branch from a20edf0 to bb4f47c Compare September 23, 2025 21:09

mhlidd requested a review from PerfectSlayer September 23, 2025 23:59

mhlidd added 9 commits September 24, 2025 17:23

adding deprecation handling

9625dd4

updating tests

a2926f9

spotless

bcb6db6

excluding json from shadowjar

31fc7b4

updating gradle files

71e309f

responding to PR comments

c65460a

updating class coverage exclude

a4de955

updating ConfigHelper to be a singleton

cad575c

refactoring ConfigHelper and ConfigurationSources to simplify code re…

5fe0a88

…adability and hide supported-configuration details

mhlidd force-pushed the mhlidd/config_inversion_base branch from 12e5514 to 5fe0a88 Compare September 25, 2025 00:25

mhlidd requested review from a team as code owners September 25, 2025 00:25

mhlidd requested review from ojung and daniel-mohedano and removed request for a team September 25, 2025 00:25

mhlidd changed the base branch from master to alexeyk/refactored-env-vars-to-be-testable September 25, 2025 00:26

mhlidd requested review from a team as code owners September 25, 2025 00:26

mhlidd requested review from robertpi and jandro996 September 25, 2025 00:26

mhlidd changed the base branch from alexeyk/refactored-env-vars-to-be-testable to mhlidd/test September 25, 2025 00:26

mhlidd changed the base branch from mhlidd/test to alexeyk/refactored-env-vars-to-be-testable September 25, 2025 00:28

mhlidd changed the base branch from alexeyk/refactored-env-vars-to-be-testable to mhlidd/test September 25, 2025 00:29

responding to PR comments and refactoring ConfigHelperTest to utilize…

4c924eb

… ControllableEnvironmentVariables

PerfectSlayer approved these changes Sep 25, 2025

View reviewed changes

...fig-utils/src/test/java/datadog/trace/config/inversion/TestSupportedConfigurationSource.java Outdated Show resolved Hide resolved

mhlidd added 2 commits September 25, 2025 11:40

updating PR comments

3bbf486

cleanup and code coverage

8ba684e

Implement Config Inversion with Default Strictness of Warning #9539

Are you sure you want to change the base?

Implement Config Inversion with Default Strictness of Warning #9539

Conversation

mhlidd commented Sep 15, 2025 • edited by bric3 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

datadog-official bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 18, 2025

Uh oh!

PerfectSlayer left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pr-commenter bot commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Debugger benchmarks

Parameters

Summary

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Implement Config Inversion with Default Strictness of `Warning` #9539

Implement Config Inversion with Default Strictness of `Warning` #9539

mhlidd commented Sep 15, 2025 •

edited by bric3

Loading

pr-commenter bot commented Sep 15, 2025 •

edited

Loading

datadog-official bot commented Sep 16, 2025 •

edited

Loading

PerfectSlayer left a comment •

edited

Loading

pr-commenter bot commented Sep 25, 2025 •

edited

Loading